In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings. The emerging field of research, known as neuroergonomics, promotes the use of wearable and portable brain monitoring sensors such as functional near infrared spectroscopy (fNIRS) to investigate cortical activity in a variety of human tasks out of the laboratory. The objective of this study was to implement an on-line passive fNIRS-based brain computer interface to discriminate two levels of working memory load during highly ecological aircraft piloting tasks. Twenty eight recruited pilots were equally split into two groups (flight simulator vs. real aircraft). In both cases, identical approaches and experimental stimuli were used (serial memorization task, consisting in repeating series of pre-recorded air traffic control instructions, easy vs. hard). The results show pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator, when completing cognitively demanding tasks. Nevertheless, evaluation of single trial working memory load classification showed high accuracy (>76%) across both experimental conditions. The contributions here are two-fold. First, we demonstrate the feasibility of passively monitoring cognitive load in a realistic and complex situation (live piloting of an aircraft). In addition, the differences in performance and brain activity between the two experimental conditions underscore the need for ecologically-valid investigations

Real-Time State Estimation in a Flight Simulator Using fNIRS

Working memory is a key executive function for flying an aircraft. This function is particularly critical when pilots have to recall series of air traffic control instructions. However, working memory limitations may jeopardize flight safety. Since the functional near-infrared spectroscopy (fNIRS) method seems promising for assessing working memory load, our objective is to implement an on-line fNIRS-based inference system that integrates two complementary estimators. The first estimator is a real-time state estimation MACD-based algorithm dedicated to identifying the pilot’s instantaneous mental state (not-on-task vs. on-task). It does not require a calibration process to perform its estimation. The second estimator is an on-line SVM-based classifier that is able to discriminate task difficulty (low working memory load vs. high working memory load). These two estimators were tested with 19 pilots who were placed in a realistic flight simulator and were asked to recall air traffic control instructions. We found that the estimated pilot’s mental state matched significantly better than chance with the pilot’s real state (62% global accuracy, 58% specificity, and 72% sensitivity). The second estimator, dedicated to assessing single trial working memory loads, led to 80% classification accuracy, 72% specificity, and 89% sensitivity. These two estimators establish reusable blocks for further fNIRS-based passive brain computer interface development

Nanospace and open-source tools for CubeSat preliminary design: review and pedagogical use-case

This paper aims to facilitate getting acquainted with CubeSat preliminary design by presenting a review of open-source tools commonly used during project first steps, and a concrete example. The light but realistic preliminary design framework is based on a real 3U CubeSat use-case, the CREME project, relying on Nanospace and a package of selected Open-Source tools. This example should allow students and non-related field experts to fully grasp the concepts needed to achieve the basics of a typical preliminary design

"Automation Surprise" in Aviation

Conflicts between the pilot and the automation, when pilots detect but do not understand them, cause “automation sur- prise” situations and jeopardize flight safety. We conducted an experiment in a 3-axis motion flight simulator with 16 pi- lots equipped with an eye-tracker to analyze their behavior and eye movements during the occurrence of such a situation. The results revealed that this conflict engages participant’s at- tentional abilities resulting in excessive and inefficient visual search patterns. This experiment confirmed the crucial need to design solutions for detecting the occurrence of conflict- ual situations and to assist the pilots. We therefore proposed an approach to formally identify the occurrence of “automa- tion surprise” conflicts based on the analysis of “silent mode changes” of the autopilot. A demonstrator was implemented and allowed for the automatic trigger of messages in the cock- pit that explains the autopilot behavior. We implemented a real-time demonstrator that was tested as a proof-of-concept with 7 subjects facing 3 different conflicts with automation. The results shown the efficacy of this approach which could be implemented in existing cockpits

EEG-engagement index and auditory alarm misperception: an inattentional deafness study in actual flight condition

The inability to detect auditory alarms is a critical issue in many do- mains such as aviation. An interesting prospect for flight safety is to understand the neural mechanisms underpinning auditory alarm misperception under actual flight condition. We conducted an experiment in which four pilots were to re- spond by button press when they heard an auditory alarm. The 64 channel Cognionics dry-wireless EEG system was used to measure brain activity in a 4 seat light aircraft. An instructor was present on all flights and in charge of initi- ating the various scenarios to induce two levels of task engagement (simple navigation task vs. complex maneuvering task). Our experiment revealed that inattentional deafness to single auditory alarms could take place as the pilots missed a mean number of 12.5 alarms occurring mostly during the complex maneuvering condition, when the EEG engagement index was high

Auditory alarm misperception in the cockpit: a preliminary ERP study to undercover evidences of inattentional deafness

Missing auditory alarms is a critical safety issue in many domains such as aviation. To investigate this phenomenon, we designed a scenario involving three flying scenarios corresponding to three different level of difficulty along with an oddball paradigm in a motion flight simulator. This preliminary study was conducted with one pilot equipped with a 32-channel EEG. The results shown that manipulating the three levels of task difficulty led respectively to rates of 0, 37, and 54% missed alarms. The EEG analyses revealed that this decrease in performance was associated with lower spectral power within the alpha band and reduced N100 component amplitude. This latter finding suggested the involvement of inattentional deafness mechanisms at an early stage of the auditory processing. Eventually, we implemented a processing chain to enhance the discriminability of ERPs for mental state monitoring purposes. The results indicated that this chain could be used in a quite ecological setting (i.e. three-axis motion flight simulator) as attested by the good results obtained for the oddball task, but also for more subtle mental states such as mental demand and stress level and the detection of target, that is to say the inattentional deafness phenomenon

A Loewner-based Approach for the Approximation of Engagement-related Neurophysiological Features

Currently, in order to increase both safety and performance of human-machine systems, researchers from various domains gather together to work towards the use of operators' mental state estimation in the systems control-loop. Mental state estimation is performed using neurophysiological data recorded, for instance, using electroencephalography (EEG). Features such as power spectral densities in specific frequency bands are extracted from these data and used as indices or metrics. Another interesting approach could be to identify the dynamic model of such features. Hence, this article discusses the potential use of tools derived from the linear algebra and control communities to perform an approximation of the neurophysiological features model that could be explored to monitor the engagement of an operator. The method provides a smooth interpolation of all the data points allowing to extract frequential features that reveal fluctuations in engagement with growing time-on-task

Blockchain-Enabled Redundant Fractionated Spacecraft System

Services and applications provided by satellites are continuously improving in terms of quality and diversity, as well as, their complexity. Resilience of traditional monolithic spacecraft is achieved mainly through redundancy, which ex- ponentially increases the complexity of their production, and therefore, their cost. One solution would be to use fragmented spacecraft systems. Redundancy is achieved by nature with local complexity. Furthermore, scalability of such a system is facilitated. However, data transfer and physical communication link between the “fragments” must be addressed. The main challenge of these systems is the management of the network of satellites. Traditional centralised networks, managed by a single entity, have been proven not to be secure, as they generate a common point of failure, susceptible to attacks. Traditional distributed networks, managed by multiple entities, require honest participants and trust between entities, restricting collab- orative mission applications. In the present article, a design for a blockchain-enabled swarm fractionated system managed by multiple trustless entities is proposed. The system is composed of functionally different nano-satellites which perform tasks from different subsystems in order to replicate the functionalities of a monolithic spacecraft. The blockchain nodes are deployed in the satellites and allow sharing of sensor data between the network. A consensus protocol ensures the validity of the shared data. The proposed system has been implemented and evaluated on a local blockchain composed of four Ethereum nodes. We believe that the proposed application opens the way to new collaborative missions between entirely trustless parties, ensuring transparency and cooperation within the system

JSatOrb: ISAE-Supaero’s open-source software tool for teaching classical orbital calculations

JSatOrb is an ISAE-Supaero’s software tool dedicated to orbital calculation and designed for pedagogical purposes, with professional level features outputs. It has been initiated to find a soft which would fill the gap between local teachers developed tools and professional tools, exploiting state of the arts algorithms concerning space mechanics calculus. Even if current provided open source libraries are not fully compliant with our pedagogical requirements (simplicity, flexibility, multi-plateform and ergonomics), they provide complete and accurate calculus methods that are dedicated to a professional use. However, GUI part is not the main concern when used by space engineers which only require API access. Concretely, JSatorb project is open-source (MIT license) and under development. It is inspired from current full-stack implementation methods. Ergonomic and intuitivity are at stack concerning the front-end, which is mainly based on Angular(https://angular.io/) and Cesium (cesiumjs.org). Efficiency and correctness on calculus are provided by the back-end part, which relies on Orekit (https://www.orekit.org/). Developed in Java, Orekit is a space dynamics open source library. It depends only on the Java Standard Edition version 8 and Hipparchus(https://hipparchus.org/) version 1

How Can Physiological Computing Benefit Human-Robot Interaction?

As systems grow more automatized, the human operator is all too often overlooked. Although human-robot interaction (HRI) can be quite demanding in terms of cognitive resources, the mental states (MS) of the operators are not yet taken into account by existing systems. As humans are no providential agents, this lack can lead to hazardous situations. The growing number of neurophysiology and machine learning tools now allows for efficient operators' MS monitoring. Sending feedback on MS in a closed-loop solution is therefore at hands. Involving a consistent automated planning technique to handle such a process could be a significant asset. This perspective article was meant to provide the reader with a synthesis of the significant literature with a view to implementing systems that adapt to the operator's MS to improve human-robot operations' safety and performance. First of all, the need for this approach is detailed as regards remote operation, an example of HRI. Then, several MS identified as crucial for this type of HRI are defined, along with relevant electrophysiological markers. A focus is made on prime degraded MS linked to time-on-task and task demands, as well as collateral MS linked to system outputs (i.e. feedback and alarms). Lastly, the principle of symbiotic HRI is detailed and one solution is proposed to include the operator state vector into the system using a mixed-initiative decisional framework to drive such an interaction